Aportació a la descripció i seguiment de camins navegables en entorns naturals a partir de l'anàlisi de regions en seqüències d'imatges

El objetivo principal fijado al inicio de esta tesis era la definición de una metodología para la descripción y seguimiento de caminos mal o débilmente estructurados (como por ejemplo los caminos de montaña, agrícolas o vecinales y las pistas forestales), orientada a un sistema de navegación autónomo. Como objetivo secundario, se había fijado el poder obtener la descripción de este tipo de entorno utilizando un sistema sensorial i computacional tan simple como fuese posible, tratando de obtener la máxima fiabilidad del sistema. Algunas de las aplicaciones donde sería de interés el sistema propuesto son el guiado de vehículos agrícolas, sistemas autónomos de extinción de incendios forestales, la ingeniería civil o la minería. La metodología que se ha seguido en la realización de la tesis se puede resumir en cuatro etapas: 1) estudio de los métodos existentes orientados a operar en otros tipos de entornos, constatando sus limitaciones para trabajar en entornos mal o débilmente estructurados. 2) análisis de que información es necesaria para navegar de forma autónoma en este tipo de entorno, y que sensores nos la pueden suministrar. 3) definición y implementación de los diferentes pasos necesarios para obtener una descripción de caminos mal o débilmente estructurados, como son el prepocesado de la imagen color, la segmentación de la imagen, la detección de obstáculos y la integración de la información presente en la secuencia de imágenes. 4) validación del método propuesto, en esta etapa se han utilizado secuencia de imágenes sintéticas, secuencia de imágenes captadas en un entorno conocido y controlado (una maqueta), y secuencia de imágenes captadas en entornos naturales. Durante la realización de esta tesis, se han analizado las diferentes alternativas que llevan a la solución del problema de la descripción del tipo de entorno considerado. Las soluciones seleccionadas están en la dirección de minimizar el coste del sistema, reducir el tiempo de proceso (y así permitir al vehículo que se desplace, dentro de las posibilidades del entorno, a mayor velocidad), garantizando, la fiabilidad de la solución adoptada. Las principales aportaciones y contribuciones realizadas a los métodos y técnicas en el ámbito de la navegación y de la visión por ordenador, resultado de la realización de esta tesis doctoral son: . Definición de un nuevo espacio de representación de la información color a partir del espacio HSI, el espacio H/I, que permite representar y analizar imágenes captadas en entornos naturales de forma eficiente. . Definición de una nueva técnica para la detección de obstáculos basada en el análisis de la evolución del tamaño de las regiones, definidas en la segmentación de la secuencia de imágenes. . Adaptación de la técnica de segmentación de crecimiento de regiones para el análisis de imágenes provenientes de escenas naturales, utilizando el color como característica. . Adaptación de un modelo para la descripción del entorno basado en matrices, para la descripción de un entorno mal o débilmente estructurado, orientado a la navegación de vehículos autónomos. Este trabajo se enmarca dentro de una temática de remarcable interés dentro de la visión por ordenador y la robótica móvil, y que también es motivo de estudio en otras universidades. Los aspectos más originales del trabajo realizado radican en la propuesta de una metodología para la descripción de caminos mal o débilmente estructurados implementable con un sistema sensorial y computacional de bajo coste, respecto otras soluciones a problemas similares.Postprint (published version

Estimating position & velocity in 3D space from monocular video sequences using a deep neural network

This work describes a regression model based on Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) networks for tracking objects from monocular video sequences. The target application being pursued is Vision-Based Sensor Substitution (VBSS). In particular, the tool-tip position and velocity in 3D space of a pair of surgical robotic instruments (SRI) are estimated for three surgical tasks, namely suturing, needle-passing and knot-tying. The CNN extracts features from individual video frames and the LSTM network processes these features over time and continuously outputs a 12-dimensional vector with the estimated position and velocity values. A series of analyses and experiments are carried out in the regression model to reveal the benefits and drawbacks of different design choices. First, the impact of the loss function is investigated by adequately weighing the Root Mean Squared Error (RMSE) and Gradient Difference Loss (GDL), using the VGG16 neural network for feature extraction. Second, this analysis is extended to a Residual Neural Network designed for feature extraction, which has fewer parameters than the VGG16 model, resulting in a reduction of ~96.44 % in the neural network size. Third, the impact of the number of time steps used to model the temporal information processed by the LSTM network is investigated. Finally, the capability of the regression model to generalize to the data related to "unseen" surgical tasks (unavailable in the training set) is evaluated. The aforesaid analyses are experimentally validated on the public dataset JIGSAWS. These analyses provide some guidelines for the design of a regression model in the context of VBSS, specifically when the objective is to estimate a set of 1D time series signals from video sequences.Peer ReviewedPostprint (author's final draft

Una propuesta de mentoría académica entre iguales para el Grado en Ingeniería Informática

En este trabajo presentamos la organización y desarrollo de un programa de mentoría académica entre iguales, Aula Lliure, y algunos de sus resultados, por el momento todavía escasos pero prometedores. Inspirado en el programa del mismo nombre de la Escuela de Ingenieros Industriales de nuestra universidad, el proyecto forma parte de un programa integral de acción tutorial dirigido, fundamentalmente, a los estudiantes de primero del Grado en Ingeniería Informática. En la puesta en marcha del proyecto han intervenido estudiantes voluntarios de cursos superiores, el equipo directivo y los servicios informáticos del centro, así como los profesores coordinadores de las distintas asignaturas impartidas. Como hemos constatado en los resultados de las encuestas realizadas, la experiencia está resultando positiva para los estudiantes que ejercen por primera vez de docentes frente a sus iguales y para los que son mentorizados. Finalmente, presentamos el análisis de los resultados obtenidos en seis asignaturas impartidas durante los cursos académicos 17/18 y 18/19.In this paper, we summarize the organization and development of an academic mentoring program among equals, Aula Lliure, and some of its results, at the moment still scarce but promising. Inspired by its namesake, still running at the Barcelona School of Industrial Engineering of our university, the project is part of a comprehensive program of tutorial action aimed, fundamentally, at first year students at the Barcelona School of Informatics. Second-year and beyond volunteer students, the management team and the computer services of the center, as well as the coordinating professors of the taught subjects, have participated in the project. As confirmed by the surveys that have been carried out, the experience is proving positive both for the students who practice for the first time as teachers in front of their peers, and for those who are mentored. Finally, we analyze the results obtained for six subjects mentored during the academic years 17/18 and 18/19.Peer ReviewedPostprint (published version

Estimation of interaction forces in robotic surgery using a semi-supervised deep neural network model

Providing force feedback as a feature in current Robot-Assisted Minimally Invasive Surgery systems still remains a challenge. In recent years, Vision-Based Force Sensing (VBFS) has emerged as a promising approach to address this problem. Existing methods have been developed in a Supervised Learning (SL) setting. Nonetheless, most of the video sequences related to robotic surgery are not provided with ground-truth force data, which can be easily acquired in a controlled environment. A powerful approach to process unlabeled video sequences and find a compact representation for each video frame relies on using an Unsupervised Learning (UL) method. Afterward, a model trained in an SL setting can take advantage of the available ground-truth force data. In the present work, UL and SL techniques are used to investigate a model in a Semi-Supervised Learning (SSL) framework, consisting of an encoder network and a Long-Short Term Memory (LSTM) network. First, a Convolutional Auto-Encoder (CAE) is trained to learn a compact representation for each RGB frame in a video sequence. To facilitate the reconstruction of high and low frequencies found in images, this CAE is optimized using an adversarial framework and a L1-loss, respectively. Thereafter, the encoder network of the CAE is serially connected with an LSTM network and trained jointly to minimize the difference between ground-truth and estimated force data. Datasets addressing the force estimation task are scarce. Therefore, the experiments have been validated in a custom dataset. The results suggest that the proposed approach is promising.Peer ReviewedPostprint (author's final draft

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Providing force feedback as relevant information in current Robot-Assisted Minimally Invasive Surgery systems constitutes a technological challenge due to the constraints imposed by the surgical environment. In this context, force estimation techniques represent a potential solution, enabling to sense the interaction forces between the surgical instruments and soft-tissues. Specifically, if visual feedback is available for observing soft-tissues’ deformation, this feedback can be used to estimate the forces applied to these tissues. To this end, a force estimation model, based on Convolutional Neural Networks and Long-Short Term Memory networks, is proposed in this work. This model is designed to process both, the spatiotemporal information present in video sequences and the temporal structure of tool data (the surgical tool-tip trajectory and its grasping status). A series of analyses are carried out to reveal the advantages of the proposal and the challenges that remain for real applications. This research work focuses on two surgical task scenarios, referred to as pushing and pulling tissue. For these two scenarios, different input data modalities and their effect on the force estimation quality are investigated. These input data modalities are tool data, video sequences and a combination of both. The results suggest that the force estimation quality is better when both, the tool data and video sequences, are processed by the neural network model. Moreover, this study reveals the need for a loss function, designed to promote the modeling of smooth and sharp details found in force signals. Finally, the results show that the modeling of forces due to pulling tasks is more challenging than for the simplest pushing actions.Peer ReviewedPostprint (author's final draft

Reflexiones sobre el diseño de una asignatura de educación en el grado de ingeniería informática

La informática es una de las áreas de conocimiento que más rápido están cambiando. Estos cambios requieren una formación continua, especializada y adaptativa. Más allá de las salidas profesionales como docente en enseñanza reglada, que requieren de formación especializada, existe la necesidad de formación y actualización de profesionales de diversos ámbitos en nuevas herramientas, procedimientos y conocimientos por parte de personas que conozcan la parte técnica y que a su vez sean capaces de realizar una formación de alta calidad. La educación es una salida profesional con cada vez más peso y que no se suele tratar en los estudios de Grado en Ingeniería Informática. Este póster presenta a la comunidad unas reflexiones sobre cómo debería ser una asignatura de educación de la informática que formara parte del plan de estudios del grado.Informatics is one of the fastest changing areas of knowledge. These changes require continuous, specialized and adaptive training. Beyond the professional opportunities as a teacher in formal education, which require specialized training, there is a need for training and updating of professionals in different fields into new tools, procedures and knowledge by people who know the technical side but who also are capable of providing high quality training. Education is, therefore, an increasingly in-demand professional opportunity for graduates in the Degree of Informatics Engineering, but it is not usually addressed into the curriculum. This poster aims to share with the community some reflections on how it should be a subject of informatics education as part of the curriculum of the undergraduate degree.Peer ReviewedObjectius de Desenvolupament Sostenible::4 - Educació de QualitatPostprint (published version

Evidence of a planktonic food web response to changes in nutrient input dynamics in the Mar Menor coastal lagoon, Spain

Nutrient input dynamics in the Mar Menor coastal lagoon has recently changed as a consequence of changes in agricultural practises. An interannual comparison of the environmental variables and the planktonic biomass size-spectra was performed between 1988 and 1997. While nitrate concentration was low in 1988, the values in 1997 increased considerably. Since 1995, two alloctonous jellyfish species (Rhyzostoma pulmo and Cotylorhiza tuberculata) occurred in large numbers in summer time and reached peak abundance in summer of 1997. The size-spectra analysis comparison revealed that, in spite of changes in nutrient input that stimulated the growth of larger phytoplankton cells, there were no significant differences in the spectra slope which followed a similar seasonal trend in both years. However, the plankton biovolume considered under the size range compared (between 2 and 1000 µm diameter) was, paradoxically, always lower in 1997. Given that there were higher nutrient levels in 1997, this finding suggest a strong top-down control mechanism of size structure. Gut contents of jellyfishes showed their preference for large diatoms, tintinnids, veliger larvae and copepods, corroborating that size structure in these assemblages can be subject to top-down control. The implication of these results is that the feeding activities of large gelatinous zooplankton (jellyfishes) may play an important role controlling the consequences of eutrophication within the Mar Menor coastal lagoon

Elaboració d’un pla de tutories per a la FIB

La majoria d'estudiants que accedeixen a la universitat, i en concret a la FIB, no són conscients del que comporta cursar una carrera universitària. Els estudis universitaris requereixen molta dedicació. Sovint els estudiants tenen mancances de coneixements, però també hàbits i actituds no adequats. En aquest sentit és interessant la figura del tutor, especialment en l’inici dels estudis, per tal de guiar i orientar els estudiants i estudiantes amb l’objectiu principal de facilitar la seva adaptació al mon universitari i millorar el seu rendiment acadèmic. Actualment les tutories són voluntàries a la FIB i hi ha pocs estudiants que demanin un tutor i pocs professors que s'impliquin en aquesta tasca. Per una banda, molts estudiants no veuen la necessitat de tenir un tutor. Per altra banda, hi ha professors que no disposen ni del material ni de la formació que seria desitjable per a fer aquesta tasca i d’altres que no creuen en la seva utilitat. L’objectiu d’aquest projecte és l’elaboració d’un Pla de Tutories per a la FIB amb les directrius i material necessari per a desenvolupar aquesta tasca. D’aquesta manera esperem captar més tutors i tutores i incidir en una millora del rendiment acadèmic.Peer Reviewe

Comparative lipidomic analysis of mammalian retinal ganglion cells and Muller glia in situ and in vitro using High-Resolution Imaging Mass Spectrometry

In order to better understand retinal physiology, alterations to which underlie some ocular diseases, we set out to establish the lipid signature of two fundamental cell types in the retina, Muller Glia and Retinal Ganglion Cells (RGCs). Moreover, we compared the lipid signature of these cells in sections (in situ), as well as after culturing the cells and isolating their cell membranes (in vitro). The lipidome of Muller glia and RGCs was analyzed in porcine retinal sections using Matrix Assisted Laser Desorption Ionization Imaging Mass Spectrometry (MALDI-IMS). Isolated membranes, as well as whole cells from primary cell cultures of RGCs and Muller glia, were printed onto glass slides using a non-contact microarrayer (Nano Plotter), and a LTQ-Orbitrap XL analyzer was used to scan the samples in negative ion mode, thereafter identifying the RGCs and Muller cells immunohistochemically. The spectra acquired were aligned and normalized against the total ion current, and a statistical analysis was carried out to select the lipids specific to each cell type in the retinal sections and microarrays. The peaks of interest were identified by MS/MS analysis. A cluster analysis of the MS spectra obtained from the retinal sections identified regions containing RGCs and Muller glia, as confirmed by immunohistochemistry in the same sections. The relative density of certain lipids differed significantly (p-value <= 0.05) between the areas containing Muller glia and RGCs. Likewise, different densities of lipids were evident between the RGC and Muller glia cultures in vitro. Finally, a comparative analysis of the lipid profiles in the retinal sections and microarrays identified six peaks that corresponded to a collection of 10 lipids characteristic of retinal cells. These lipids were identified by MS/MS. The analyses performed on the RGC layer of the retina, on RGCs in culture and using cell membrane microarrays of RGCs indicate that the lipid composition of the retina detected in sections is preserved in primary cell cultures. Specific lipid species were found in RGCs and Muller glia, allowing both cell types to be identified by a lipid fingerprint. Further studies into these specific lipids and of their behavior in pathological conditions may well help identify novel therapeutic targets for ocular diseases.This study was supported by the grants RETOS MINECO FEDER (RTC-2016-48231), PUE 2018-1-0004, UPV/EHU PPGA 18/18 and Elkartek KK-2019/00086 to E.V